Web Survey Bibliography
Do survey respondents, recruited with extraordinary efforts, provide answers of lower quality than respondents who are recruited more easily? This question has worried survey practitioners and analysts alike for at least four decades (Cannell and Fowler 1963; Robins 1963), but an answer is not known. Although no direct relationship exists between response rates and nonresponse bias (Groves and Peytcheva 2008), an open question is the relationship between efforts to increase response rates and other sources of survey error, in particular, measurement error. The common hypothesis is that those who require greater recruitment effort provide answers of lower quality than those who are recruited more readily. A latent cooperation continuum often is posited (e.g., (Mason, Lesser, and Traugott 2002; Cannell and Fowler 1963), such that those who are most difficult to recruit to the sample pool have the lowest motivation and are thus the worst reporters. However, it is not clear if this is generally the case.
This paper reviews existing literature on the relationship between the levels of effort exerted for sample member recruitment and data quality, with a primary focus on item nonresponse. Two methods – a quantitative meta-analysis and a systematic qualitative review – are used to examine 44 articles examining the relationship between levels of recruitment effort and measurement error. These studies use five different measures of levels of effort (number of contact attempts/follow-up reminders, refusal conversion, date of interview, combination of these three, estimated response propensity) and look at multiple measurement error indicators (e.g., item nonresponse, response accuracy, signed deviations, scale reliability, acquiescence, non-differentiation).
This review asks the following four questions.
1.Do respondents recruited with more effort have higher item nonresponse rates than respondents recruited more easily?
2.Are particular study characteristics associated with higher item nonresponse rates among respondents recruited with more effort relative to respondents recruited with less effort?
3.Are particular types of items associated with higher item nonresponse rates among respondents recruited with more effort relative to respondents recruited with less effort?
4. Is there consistent evidence about higher or lower levels of other types of measurement error, and does it vary by the level of effort measure?
Conference homepage (abstract)/(full text)
Web survey bibliography (4086)
- Comparison of Web-Based versus Paper-and-Pencil Self-Administered Questionnaire: Effects on Health Indicators...; 2008; van de Looij-Jansen, P. M., de Wilde, E. J.
- By the Numbers: The pros and cons of sampling modes; 2008; Piekarski, L.
- A look at the impact of boredom on the respondent experience; 2008; Puleston, J., Sleep, D.
- Does speed in completing an online questionnaire have an influence on its reliability?; 2008; Montag, C., Reuter, M.
- To Mix or Not To Mix Survey Modes; 2008; Dillman, D. A.
- Finding Lost Respondents on MySpace and Facebook; 2008; Claggett, A.
- Blast from the Past: The Challenges of Gaining Cooperation for a Follow-up Survey Several Years Later...; 2008; Costanzo, D., Sipulski, E., Carris, K., O'Connor, K.
- Data Collection on Handheld Computers with Voice Recognition Technology; 2008; Malakhoff, L.
- Web Survey Technology; 2008; Boucher, D.
- The New GSS Data Dissemination Website; 2008; Cooke, M.
- Item Non-Response: Does Mode (Web versus Paper) Make a Difference?; 2008; Payn, B.
- How To Write Better Survey Questionnaires; 2008; Triplett, T. A.
- The Use of the Internet in the American Community Survey Group Quarters Frame Creation; 2008; Williams, A.
- Technology and the Survey Interview and Questionnaire; 2008; Couper, M. P.
- Response Effects in Video-enhanced Web Surveys; 2008; Fuchs, M.
- Evaluation of Sample Designs for Telephone Surveys That Include Cell Phones; 2008; Steeh, C. G.
- Costs and Benefits from Conducting a Supplemental Sample of Cell Phone Numbers in National RDD Surveys...; 2008; Kennedy, C.
- Cell Phone Interviewing; 2008; Lee, R. H.
- Reaching Cell Phone Only Households; 2008; Anatchkova, B.
- Interviewing with Cell Phones: Trends and Effects on Data Quality and Operation Costs; 2008; Leissou, E., Gouskova, E., McGonagle, K., Beaule, A., Stafford, F.
- Text Messaging as a Data Collection Tool; 2008; Nordstrom, P.
- Swedish National Data Service's Strategy for Sharing and Mediating Data; 2008; Carlhed, C., Alfredsson, I.
- A Web-Based Survey of the Relationship Between Buddhist Religious Practices, Health, and Psychological...; 2008; Wiist, W. H., Sullivan, B. M., Wayment, H. A., Warren, M.
- Social Research Methods; 2008; Bryman, A.
- The Good, the Bad, and the Ugly of Cell Phone Surveying; 2008; Barron, M.
- Experiments Exploring the Relationship (or Lack Thereof) Between Nonresponse Error and Measurement Error...; 2008; Tourangeau, R., Groves, R. M., Presser, S., Toppe, C., Kennedy, K., Yan, T.,
- Using Audio Recording Using Audio to Detect Data Collection Error; 2008; Thissen, R., Nguyen, M.
- When do nonresponse follow-ups improve or reduce data quality? A meta-analysis and review of the existing...; 2008; Olson, K., Feng, C., Witt, L.
- Respondent Incentives in a Multi-Mode Panel Survey: Cumulative Effects on Non-Response and Bias; 2008; Jaeckle, A., Lynn, P.
- A comparison of results from an alcohol survey of a prerecruited Internet panel and the National Epidemiologic...; 2008; Heeren, T., Edwards, E., Dennis, J. M., Rodkin, S., Hingson, R. W., Rosenbloom D. L.
- Effects of Pre-coding Response Options for Five Point Satisfaction Scale in Web Surveys; 2008; Callegaro, M., Wells, T., Kruse, Y.
- ...; 2008; Dillman, D. A., Smyth, J. D., Christian, L. M., O'Neill, A. C.
- Accommodating New Technologies: Mobile and VoIP Communication; 2008; Piekarski, L., Steeh, C. G.
- Internet Technology and Social Capital:How the Internet Affects Seniors' Social Capital and Wellbeing...; 2008; Sum, S., Mathews, M. R., Pourghasem, M., Hughes, I.
- Quality Criteria for Quantitative, Qualitative and Mixed Methods Research: A View from Social Policy...; 2008; Bryman, A., Becker, S., Sempik, J.
- Encyclopedia of Survey Research Methods; 2008; Lavrakas, P. J.
- An implementation of a within-household selection procedure for web surveys; 2008; Callegaro, M., Osborn, L., Debell, M., Leuvano, P.
- Response options order effect and category number association: An experiment using items on a five point...; 2008; Tang, G., Callegaro, M.
- More than the digital divide?: Investigating the differences between Internet and non-Internet users; 2008; Zhang, C., Callegaro, M., Thomas, M.
- Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method ; 2008; Dillman, D. A., Christian, L. M., Smyth, J. D.
- Enhancing the Travel Survey Process and Data Using the CATI System ; 2008; Morency, C.
- Mobile Research in Marketing: Design and Implementation Issues; 2008; Li, H., Townsend, L.
- Is data collection via web a valid methodology?; 2008; Otero, P.
- Identifying Multiple Submissions in Internet Research: Preserving Data Integrity ; 2008; Bowen, A. M., Daniel, C. M., Williams, M. L., Baird, G. L.
- The Methodological Issues Associated With Internet-Based Research; 2008; Beddows, E.
- A national Internet survey on rapid sequence intubation patterns from Turkey ; 2008; Guneysel, O., Onur, O. E., Akoglu, H., Eroglu, S., Denızbası, A.
- Experiments in Mobile Web Survey Design; 2008; Peytchev, A., Hill, C.
- A multimedia-based and time-sensitive interactive web survey for road user opinion on traffic condition...; 2008; Posawang, P., Phosaard, S., Polnigongit, W., Pattara-Atikom, W.
- The Effectiveness of CATI Method in the Survey “Structure of Agricultural Holdings”; 2008; Jakóbik, K., Ruta, G.
- CAQDAS, Secondary Analysis and the Coding of Survey Data; 2008; Fielding, N.